Optimal $\gamma$ and $C$ for $\epsilon$-Support Vector Regression with RBF Kernels

The objective of this study is to investigate the efficient determination of C and γ for Support Vector Regression with RBF or mahalanobis kernel based on numerical and statistician considerations, which indicates the connection between C and kernels and demonstrates that the deviation of geometric distance of neighbour observation in mapped space effects the predict accuracy of -SVR. We determinate the arrange of γ & C and propose our method to choose their best values. Introduction Traditional forecasting algorithm like ARIMA, Exponential Smoothing can provide good forecasting results with regard to trend, season and other linear correlated features . In practice, those features are normally non-linear. To solve non-linear forecasting problems, -Support vector regression ( -SVR) is employed. Support vector machine (SVM) is nowadays wildly used for classification problems in many areas. However, -SVR is hardly used because of the uncertain parameter C, for its dual problem. By using RBF or Mahalanobis kernels, value of γ decides determination of kernel matrix and hence is the key of whole system. An overview of choosing those parameters is given by [1]. Best selection of C is given by [6], which can be done with in limited iterations. The most used methods are searching methods like random search, grid search, pattern search. Cherkassy and Ma (2004)[2] have proposed one way to determinate those parameters directly from the data. But there is still one parameter c need to be extra searched within pre-existing arrange. Our propose is also to determinate C and γ directly but without using any searching methods. In this paper, we give a short overview of -SVR in section 1 and discuss the determination of C and γ in section 2 and 3. Then we test our algorithms using practice data in section 4 and summery results in section 5. 1 -Support Vector Regression Assume (x1, y1), (x2, y2), ..., (xN , yN), . . . ∈ X × R ⊆ R are observation pairs, xi ∈ X ⊂ R is feature vector and yi ∈ R is the target output. Define NX as the total number of all observation in training set. According to [3] 2 the dual problem form of -SVR under given C with kernel K is min 1 2 (α− α∗)TQ(α− α∗) + ∑ (αi + α ∗ i ) + ∑ yi(αi − α∗ i ) subject to e (α− α∗) = 0, 0 ≤ αi, α∗ i ≤ C, i = 1, ..., Ns, (1) where Qi,j := K(xi, xj) and α = (α1, ..., αNs), α ∗ = (α∗ 1, ..., α ∗ Ns). The corresponding approximate function is